Europafsches Patentamt 
European Patent Office 
Office eu rope en des brevets 



© Publication number: 



0 468100 A1 



EUROPEAN PATENT APPLICATION 



® Application number: 90125596.8 
® Date of filing: 27.12.90 



<£) lnt.Cl.S:G01N 15/14, G01N 33/48, 
G01N 21/05 



© Priority: 24.07.90 JP 195934/90 


© Applicant: TOA MEDICAL ELECTRONICS CO., 




LTD. 


© Date of publication of application: 


2-1, Minatojimanakamachl 7-chome Chuo-ku 


29.01.92 Bulletin 92/05 


Kobe-shi Hyogo-ken(JP) 


© Designated Contracting States: 


© Inventor: Kosaka, Toklhiro 


DE FR GB IT 


462-81, Ishlmori, Kanno-cho 




Kakogawa-shi, Hyogo-ken(JP) 




© Representative: Lehn, Werner, Dipl.-lng. et al 




Hoffmann, Eitle & Partner Patentanwalte 




Arabellastrasse 4 




W-8000 Munchen 81 (DE) 



© Automatic focal-point adjustment method in flow Imaging cytometer. 

© A flow imaging cytometer for imaging the dimensions and shapes of particle components in a flow of a 
specimen solution employs a separate control solution, which is for managing precision, containing standard 
particles. A method of adjusting focal point with respect to the specimen flow includes steps of imaging the 
standard particles in the control solution, calculating an evaluation parameter which represents the definition of 
the image, and moving the flow cell or associated optical system so as to maximize the value of this parameter. 
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This invention relates to flow imaging cytometry in which a specimen such a blood or urine suitably 
stained is introduced to a flow cell to form a flat, sheathed flow within the cell, the sheathed flow zone is 
irradiated with strobe light, and a cell image obtained by a video camera is analyzed by image processor. 
More particularly, the invention relates to an automatic focal-point adjustment method used in such flow 
5 imaging cytometry. 

A cell analyzing apparatus for imaging cells flowing in the form of a flat, sheathed stream and 
automatically classifying and counting the cells utilizing image processing technology has been disclosed in 
the specifications of Japanese Patent Application Laid-Open (KOKAI) No. 57-500995 (1982) and US Patent 
No. 4,338,024. 

10 An automatic focusing method is also known in which the image of an object produced by a lens is 
formed in an automatically focused state at a prescribed position. For example, the specification of 
Japanese Patent Publication (KOKOKU) No. 42-14096 (1967) describes a technique which uses an array of 
photoelectric elements having a very small surface area. Differences in output between mutually adjacent 
ones of the photoelectric elements are totaled and the total value is maximized when focusing is achieved. 
75 This technique detects the maximum point. Also known are various evaluation relations for evaluating the 
definition of an image. For example, the specification of Japanese Patent Publication (KOKOKU) No. 58- 
41485 (1983) describes making use of a mean-square function. 

Since cameras such as still cameras or video cameras deal with images in which there is almost no 
change from one frame to the next, performing a focal-point adjustment while moving lenses or a group of 
20 image pickup elements in very small amounts is comparatively easy. However, in capturing a cell image or 
the like in flow imaging cytometry, the image changes every frame (every 1/30 of a second), and the 
positions and number of cells that appear in each frame change vastly. Consequently, performing focusing 
by comparing the same image from one frame to the next while moving a lens or the like in minute 
increments is difficult. Accordingly, automatic focusing is not easy to achieve. 
25 An object of the present invention is to provide a efficient automatic focal-point adjustment method used 
in flow imaging cytometry and adapted to solve the aforementioned problems encountered in cell analysis. 

According to the present invention, the foregoing object is attained by providing an automatic focal- 
point adjustment method in a flow imaging cytometer in which a specimen solution containing particle 
components such as cells is made to flow as a flat specimen solution having a sheathing liquid as an outer 
30 layer thereof along a flat flow path within a flow cell, light irradiating means and imaging means are 
arranged on opposite sides of the flow cell, a still image of the specimen solution is captured and the image 
is subjected to image processing, whereby analysis such as classification and enumeration of the particle 
components contained in the specimen solution is performed. The automatic focal-point adjustment method 
is characterized by passing a control solution, which contains particles of uniform dimensions and shape, 
35 through the flow cell as a dummy solution, moving the position of the flow cell or of an optical measuring 
component by a fine adjustment device such as a stepping motor, continuously capturing the control 
solution during such movement and calculating evaluation parameters which accurately represent the 
definition of the images obtained, and adjusting the position of the flow cell or of the optical measuring 
component in such a manner that a value of an evaluation parameter attains a maximum value, thereby 
40 eventually achieving focal-point adjustment. 

The figures are views for describing an embodiment of the present invention, in which: 
Figs. 1(a) and (b) illustrate examples of original image data and Sobel operators and Fig. 1(c) Laplacian 
operators when obtaining evaluation parameters of an image; 
Fig. 2 is an example of an evaluation parameter curve; 
45 Fig. 3 is a block diagram showing an image processing system for flow imaging cytometry; 

Fig. 4 is a block diagram of a morphology processor board ideal for use in the system of Fig. 3; 
Fig. 5 is an example of a histogram of an evaluation parameter; 
Fig. 6 is a basic flowchart of focus adjustment; and 
Fig. 7 is a basic flowchart of coarse focus adjustment. 
50 An embodiment of the present invention will now be described in detail with reference to the 
accompanying drawings. 

In flow imaging cytometry according to the present invention, a cumulative value obtained by totaling 
differential values at local areas of an image over the entire frame of the image is used as an evaluation 
parameter representing the degree of focusing of the image, such as the image of a cell. Sobel operators 
55 shown in Fig. 1(b) are employed as an example of operators for obtaining a differential value in a local area 
of the image. As a coefficient corresponding to each item of data of 3 x 3 = 9 pixels surrounding a pixel of 
interest x Vi shown in Fig. 1(a), use is made of a new value corresponding to the pixel of interest xy using the 
values of nine pixels obtained by multiplying the above-mentioned pixel by horizontal and vertical Sobel 
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operators. This is referred to as a 3 x 3 convolution. Such new values are calculated with regard to all 
pixels in the entire image frame. 

More specifically, new values are calculated in accordance with the following as differentiated values in 
the horizontal and vertical directions of the pixel of interest: 
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In a background portion where particles do not appear the new value is approximately zero. However, when 
20 a particle enters the image frame, the calculated value at the contour thereof becomes larger in proportion 

to contrast as focusing is performed. Accordingly, a value obtained by accumulating such calculated values 

over the entire image frame can be utilized as an evaluation parameter indicating the degree of focusing. 
According to the present invention, a solution containing control particles for precision management in 

which particles are uniform in dimensions and shape is introduced to a flow cell, and focusing is performed 
25 by processing the image in which the particles appear, thereby raising the precision of classification and 

analysis of a specimen containing a wide variety of particle components. In this case, the number of 

particles which appear in one frame will be only several tens at low magnification and several at high 

magnification. As a result, the cumulative differential value for one frame will exhibit a large statistical 

variance when compared with other frames. 
30 Accordingly, a value obtained by totaling the cumulative differential value for one frame over several 

frames or several tens of frames is adopted as the final evaluation parameter, denoted by P. This parameter 

P is given by the following formula: 



35 



P = 2 2 {(Sbx) 2 /k + (Sby) 2 /k} 

frame ij 



(3) 



40 



Represents entirety of one frame 
Represents several frames or 
several tens of frames 



In the above formula for calculating the evaluation parameter, taking the square of Sbx, Sby is effective 
in order to raise the sensitivity and Q value of the degree of focusing. The coefficient k is given a value 
45 which will prevent overflow. 

Another method of obtaining the evaluation parameter P is to use Laplacian operators. Two types of 
Laplacian operators (c-1), (c-2) are illustrated in Fig. 1(c). When the Laplacian operator (c-1) is used, a 3 x 
3 convolution takes on the following value: 



50 
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lapx.. = 4x. - x i _ 1 - x. - x. - x. +1 ... (4) 
When the Laplacian operator (c-2) is used, we have 
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laPX ij = 8X i " X i-1 " X i-1 " X i-1 " X i 
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. . . (5) 



If the evaluation parameter P is obtained using Equation (4) or (5), P will be expressed by either of the 
following: or 

P = 2 2 llapx.jl/h ... (6) 



frame ij 



P = 2 2 (lapx. )Vh ... (7) 

frame ij 1J 

25 where h is a constant. 

In accordance with the Sobel method, the differential value Sbxy which uses the horizontal operator and 
the differential value Sbyij which uses the vertical operator are obtained, and the evaluation parameter P is 
calculated using Equation (3). With the Laplacian method, however, it will be noted that one step of the 
calculations can be omitted. 

30 The above-mentioned evaluation parameter is obtained by real-time processing of the image data using 
an image processor with which the cytometer is furnished. When the evaluation parameter is plotted while 
finely moving the flow cell or lens with regard to one frame, a curve of the kind shown in Fig. 2 is obtained. 
The point at which the maximum value is obtained is the position at which focusing is achieved. An 
evaluation parameter according to Equation (3) is obtained using an image processor shown in Fig. 3. An 

35 example of the circuitry of an image processor board forming the nucleus of Fig. 3 is illustrated in Fig. 4. A 
morphology processor board is a board on which general functions necessary for processing a cell image 
are mounted; the board is not specially mounted in order to focus an image. Illustrated below is a method of 
obtaining evaluation parameters of a plurality of frames in which one field (1/60 of a second) is adopted as 
one step. 

40 In Figs. 3 and 4, (i) data indicative of the frame of an odd-numbered field A/D-converted by an A/D 
board, namely a video signal analog/digital conversion board, is inputted to a process channel 2 of a 
morphology processor board MP via an image data bus C, a differential value (Sbx^/k in the horizontal 
direction is obtained in real-time, and this data is stored in a working memory WM via an image data bus E; 
(ii) data indicative of the frame of an even-numbered field A/D-converted by the A/D board is inputted to the 

45 process channel 2 of the morphology processor board MP via the image data bus C, a differential value 
(Sby) 2 /k in the vertical direction is obtained, this data is inputted to a process channel 1 via the image data 
bus E, the data indicative of (Sbx) 2 /k stored in the working memory is read out to the image bus 
simultaneously, the data is added to an ALU, and a histogram of the added values is prepared in real-time 
by a histogramer; and (iii) the results of the histogram of the kind shown in Fig. 5 prepared in step (ii) are 

so accumulated by a digital signal processor DSP on the digital signal processor board, and an evaluation 
parameter for one frame is obtained. 

If a general-purpose high-speed microprocessor or digital signal processor is used, calculating the 
evaluation parameter for one frame will take one second assuming that one frame is composed of 256 x 
256 pixels. Therefore, about 20 seconds will be required to obtain evaluation parameters for 20 frames. In 

55 actuality, however, a longer time is required since focus adjustment is performed while moving the lenses or 
flow cell. On the other hand, if an evaluation parameter is obtained in real-time with respect to one frame 
which appears every 1/30 of a second, 20/30 of a second will be required to obtain evaluation parameters 
for 20 frames. Ideally, therefore, an evaluation value for focusing should be obtained using a digital filter, 
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namely a 3 x 3 convolution filter, described below, which incorporates a high-speed multiplier and a high- 
speed cumulative adder. At the same time, processing similar to that of step (i) is executed with regard to 
the next frame in conjunction with step (iii). (iv) Processing similar to that of step (ii) is executed with regard 
to the next frame, (v) After processing similar to that of step (iii) is executed with regard to the next frame, 

5 the evaluation parameter of the initial frame and the evaluation parameter of the next frame are added. At 
the same time, processing similar to that of step (i) is executed with regard to the next frame. Thereafter, 
processing is applied to successive frames one after another in the same manner. 

Since one frame is the sum of the odd-numbered and even-number fields, the differential values Sbx, 
Sby of Equation (3) can be obtained in real-time by an FIR filter used as the 3 x 3 convoluter (Fig. 4) of 

70 process channel 2. Furthermore, processing for squaring both differential values and then multiplying them 
by 1/k can be performed in real-time by a look-up table LUT3. 

An actual focal-point adjustment algorithm using the evaluation parameter P will now be described. A 
system whose basic aim is to perform accurate focal-point adjustment in an efficient manner and which 
requires imaging while changing magnification will be described in accordance with Fig. 6 showing a 

75 focusing flowchart. Focusing in the present invention is divided into two stages, namely, a coarse 
adjustment step and a fine adjustment step. 

First, a control solution is passed through a flow cell and imaged by a video camera, and an initial value 
Poz of the evaluation parameter of the image on the low-magnification mode is obtained. In this case, 
cumulative differentiated values for every several frames are obtained while the flow cell or lens is finely 

20 adjusted. When fine adjustment of focus is performed, the speed at which the flow cell or lens is moved 
cannot be made very high, though the speed of movement can be raised at coarse adjustment of focus. 
When it is determined that an image is too far out of focus, namely when the relation P ot < th t holds with 
regard to a threshold th t that gives the boundary between a coarse-adjustment region and a fine-adjustment 
region, it is required to bring adjustment to the fine-adjustment region much more speedily by following 

25 Step 1, which constitutes performing coarse adjustment at a high speed of flow cell or lens movement. 

The threshold value th t is decided as follows: If the apparatus to be focused is undergoing the focus 
adjustment for the first time, the first step is to obtain the mean value and mean rate of fluctuation of the 
evaluation parameters which prevail when the temperatures and sheath pressures of several sets of the 
same model apparatus already adjusted are varied to the maximum limit within a practical range. A value 

30 obtained by multiplying this mean value by 0.9, for example, is adopted as the initial value of th t . Once an 
apparatus has already had its focus adjusted, the value of th t when the apparatus is readjusted will be 
obtained in accordance with the following equation using an evaluation parameter value P pl that prevailed at 
the initial adjustment or immediately preceding adjustment: 

35 th t = P pl (1 - mean fluctuation rate) x 0.9. 

By thus deciding th t , focus readjustment of an apparatus already focused once starts, in most cases, 
from the fine adjustment of Step 2, with coarse adjustment being skipped. 
40 The focal-point adjustment process of the present invention, which is premised on the foregoing, is 
summed up by the flowchart of Fig. 6. Each procedure will now be described. 

(1) The control solution is passed through the flow cell. 

(2) The initial value P ol of the evaluation parameter on the low-magnification mode is obtained. 

(3) The initial value P M of the evaluation parameter and a predetermined value th x are compared. If P ot 
45 < th t holds, the program proceeds to Step 1; if P Q i £ th t holds, the program skips Step 1 and proceeds 

to Step 2. 

(4) Step 1 : Coarse adjustment of focus on low-magnification mode 

The evaluation parameter P n is obtained while moving the flow cell, and the program proceeds to 
Step 2 if P n £ th t holds, 
so (5) Step 2: Fine adjustment of focus on low-magnification mode 

The evaluation parameter P n is obtained while moving the flow cell, and the flow cell is moved to a 
position at which the evaluation parameter P n is maximized. 
(6) Step 3: Fine adjustment of focus on high-magnification mode, and verification 
Fig. 7 illustrates an adjustment algorithm to which the foregoing adjustment procedure is capable of 
55 being applied through Steps 1, 2, 3. This deals also with an anomaly which occurs when an adjustment 
error is determined with regard to an image in which about 180 particles appear in six frames. The 
adjustment time required for Step 1 is 0.5 - 1 1 seconds. 

Next, when the apparatus is slightly out of focus at low magnification, Step 2 is executed in the next 
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procedure in the above-described adjustment algorithm. The time required for adjustment at Step 2 is 6 - 7 
seconds. 

Step 2 (Fine Adjustment): P G £ th t 

5 

(1 ) P n for every six frames is obtained while moving the flow cell continuously It/5 per six frames in the 
right-hand direction. 

(2) If the relation P n < th t is attained and P n is obtained for four or more points, then continuous 
movement in the right-hand direction is stopped. 

w (3) P n for every six frames is obtained while moving the flow cell continuously it/5 per six frames in the 
left-hand direction in a manner similar to (1). 

(4) If the relation P n < th t is attained, then continuous movement in the left-hand direction is stopped. 

(5) The value of P at each point obtained in (1) and (3) is subjected to smoothing processing and the 
amount of backlash is found. The point at which P is maximized is obtained. 

75 (6) Upon taking backlash into consideration, the flow cell is moved to the point obtained in (5). 

(7) The value of P in 30 frames at this position is obtained, and this value P pI and the position P sr are 
stored in memory. 

(8) When P pt is smaller than a prescribed value P plS) this is determined to be an adjustment error 2. 

(9) The program proceeds to Step 3. 

20 Next, following the focus adjustment on the low-magnification mode, Step 3, which is for a fine 
adjustment on the high-magnification mode, is executed through the following procedure: 

Step 3: Fine Adjustment on High-Magnification Mode 



25 (1 ) Magnification is changed over from the low mode to the high mode, and 30 frames of P Q h at the point 
adjusted on the low-magnification mode is obtained. (There are about 60 particles in 30 frames.) 

(2) The relation P oh £ th h (the threshold on the high-magnification mode) is verified. If the relation P oh < 
th h holds, then this is determined to be an adjustment error 3. 

(3) Pnh for every 30 frames is obtained while moving the flow cell continuously Jtth/5 per 30 frames in the 
30 right-hand direction. 

(4) If the relation P nh < th h is attained, then continuous movement is stopped. 

(5) Pnh for every 30 frames is obtained while moving the flow cell continuously Jtth/5 per 30 frames in the 
left-hand direction in a manner similar to (3). 

(6) The value of P at each point obtained in (5) is subjected to smoothing processing and the point at 
35 which P is maximized is obtained. 

(7) Upon taking backlash into consideration, the flow cell is moved to the point obtained in (6). 

(8) The value of P in 90 frames at this position is obtained, and this value P ph and the position P sh are 
stored in memory. 

(9) When P pn is smaller than a prescribed value P phs , this is determined to be an adjustment error 4. 

40 (10) Magnification is changed over from the high mode to the low mode, and 30 frames of P ot at the 
point adjusted on the high-magnification mode is obtained. 

(11) Pot is compared with P pt obtained in Step 2 and an adjustment error 5 is determined when P oi < 
k 2 P p i is found to hold (k 2 = 0.8 - 0.85). 

(12) P P |, P ph , P sl and P sh which prevailed at the time of initial focus adjustment, two focus adjustments 
45 previously, the preceding focus adjustment and the present focus adjustment are compared and 

checked. When the differences exceed a prescribed value, an adjustment error 6 is determined. 
The time required for the adjustment of Step 3 is 28 - 32 seconds. 

The focus adjustment errors which occur in the foregoing steps can be considered to be caused by the 
following: 
50 (1) Adjustment Error 1 

a) an abnormality in the drive mechanism using the stepping motor; 

b) the flow of control particles into the flow cell is not normal owing to an abnormality in the fluid 
system; and 

c) clogging of a nozzle, etc. 
55 (2) Adjustment Error 2 

a) the position of the sample stream flowing through the flow cell has been disturbed; 

b) the thickness of the sample stream flowing through the flow cell is excessive; and 

c) the strobe has deteriorated. 
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(3) Adjustment Error 3 

a) the position of the center of the sample stream flowing through the flow cell fluctuates in the 
thickness direction between the point at low magnification and another point at high magnification; and 

b) the thickness of the sample stream flowing through the flow cell is excessive. 
5 (4) Adjustment Error 4 

a) the thickness of the sample stream flowing through the flow cell is excessive; and 

b) the strobe has deteriorated. 

(5) Adjustment Error 5 

the position of the center of the sample stream flowing through the flow cell fluctuates in the thickness 
70 direction between the point at low magnification and the point at high magnification. 

(6) Adjustment Error 6 

a) the flow cell or lens drive unit is loose and worn; 

b) the strobe has deteriorated; 

c) dust has become attached to the lens, flow cell, etc.; and 
75 d) the inner wall surface of the flow cell is contaminated. 

In accordance with the adjustment method of the present invention, a still image of a specimen solution 
is captured every 1/30 of a second, and therefore identical frames do not occur. Evaluation parameters are 
obtained using these images that change from one instant to the next. Accordingly, though the evaluation 
parameters differ greatly from frame to frame, if a value obtained by adding up the evaluation parameters 
20 for a plurality of frames is adopted as the evaluation parameter, then the differences among a plurality of 
frames will be absorbed according to the rules of algebra, the characteristic features of the image are made 
to stand out and this is accurately reflected in the focused state of the cell image. 

The value of the evaluation parameter is obtained in real-time while continuously moving the flow cell or 
lens, and the position at which this value is maximized is rapidly determined. In addition, the time required 
25 for adjustment can be curtailed first by coarse adjustment of focus at low magnification and then by fine 
adjustment of focus at high magnification. 

If the maximum value of the evaluation parameter obtained whenever focal point is adjusted is stored in 
memory, any abnormality in the system can be discovered by monitoring and fluctuation in the stored 
value. 

30 The present invention provides the following advantages: 

(1) An evaluation parameter is obtained using images from a plurality of frames, during which time 
control particles for managing precision are used in focal-point adjustment, thereby making it possible to 
perform coarse adjustment and fine adjustment efficiently and precisely in a comparatively short period 
of time. 

35 (2) A magnification change-over step is additionally provided, coarse adjustment of focus is performed 
first on the low-magnification mode where a large number of particles appear in one frame, and then fine 
(precise) adjustment of focus is performed on the high-magnification mode. This makes it possible to 
raise efficiency by a wide margin. 

(3) The hardware of a cell analyzing system for processing cell images can be utilized as is. In other 
40 words, it is unnecessary to add separate hardware for adjusting focal point. As a result, the advantages 
mentioned above can be realized while minimizing cost. 

As many apparently widely different embodiments of the present invention can be made without 
departing from the spirit and scope thereof, it is to be understood that the invention is not limited to the 
specific embodiments thereof except as defined in the appended claims. 

45 

Claims 

1. In a flow imaging cytometer in which a specimen solution containing particle components is made to 
flow while sheathed by a sheathing liquid through a flat flow passageway of a flow cell, still images of 
so the specimen solution are captured by light irradiating means and image pick-up means arranged on 
opposite sides of the flow cell, and analysis of particle components contained in the specimen solution 
is performed by image processing, a focal-point adjustment method comprising the steps of: 

introducing a control solution, which contains particles of uniform dimensions and shape, to the flow 

cell; 

55 capturing still images of the control solution while moving a component of an optical measuring 

system such as the flow cell; 

calculating evaluation parameters which represent the definition of the still images; and 
performing focal-point adjustment with respect to specimen flow by adjusting the position of said 
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component in such a manner that a value of an evaluation parameter is maximized. 

The method according to Claim 1 further comprising the steps of: 

comparing an initial value P 0 of said evaluation parameter and a predetermined reference value th 
when the control solution is introduced to the flow cell; 

obtaining an evaluation parameter P n and establishing a state in which P n > th holds by moving the 
flow cell when P 0 < th holds; and 

setting position by moving the flow cell to a position at which the evaluation parameter P n is 
maximized. 

The method according to Claim 2, wherein the position of an optical component such as a lens system 
of the flow cytometer is moved instead of the position of the flow cell. 

The method according to any one of Claims 1 through 3 further comprising the steps of: 

first calculating said evaluation parameter on a low-magnification mode of the optical system; and 
when said evaluation parameter has been maximized, maximizing the evaluation parameter on a 

high-magnification mode of the optical system. 

The method according to any one of Claims 1 through 4 further comprising the steps of: 
storing a maximum value of said evaluation parameter; and 

sensing an abnormality of the measurement system by monitoring a fluctuation in the maximum 
value of said stored evaluation parameter. 
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OBTAIN EVALUATION PARAMETER Pol 
ON LOW- MAGNIFICATION SIDE 



Tno 

STEP 1 i 


PERFORM COARSE ADJUSTMENT OF FOCUS 
ON LOW- MAGNIFICATION SIDE 




STEP 2 






PERFORM FINE ADJUSTMENT OF FOCUS 
ON LOW- MAGNIFICATION SIDE 





STEP 3 j 

PERFORM FINE ADJUSTMENT OF FOCUS 
ON HIGH-MAGNIFICATION SIDE. RETURN 
TO LOW-MAGNIFICATION SIDE, AND CHECK 
AGAIN 
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